Skip to content

Add Performance category to the changelog and release notes#16065

Merged
mtreinish merged 3 commits intoQiskit:mainfrom
eliarbel:performance-cagetory
Apr 30, 2026
Merged

Add Performance category to the changelog and release notes#16065
mtreinish merged 3 commits intoQiskit:mainfrom
eliarbel:performance-cagetory

Conversation

@eliarbel
Copy link
Copy Markdown
Member

This PR adds a new category to the release notes and changelog for reporting performance improvements.

Details

  • I've put the performance category to appear after all the additions, changes, deprecations and issues sections in the release notes.
  • Added a new Changelog: Performance label to the repo (directly in Github). There is also the preexisting performance label, which is currently used by several PRs. We should review which PRs should switch to the new label once this PR is merged and then add release notes for those which are marked with Changelog: None.

AI/LLM disclosure

  • I didn't use LLM tooling, or only used it privately.

@eliarbel eliarbel requested a review from a team as a code owner April 20, 2026 11:11
@qiskit-bot
Copy link
Copy Markdown
Collaborator

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core

Comment thread releasenotes/config.yaml Outdated
Copy link
Copy Markdown
Member

@jakelishman jakelishman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's much in the 2.5 branch yet that should get recategorised, but it might be worth having a look after this PR merges, before we forget.

@eliarbel eliarbel added the Changelog: None Do not include in the GitHub Release changelog. label Apr 20, 2026
Copy link
Copy Markdown
Member

@mtreinish mtreinish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're adding a dedicated section on this how is this distinct from a feature? Like to me "X is 25% faster because of internal refactoring" is a feature. So I guess the question is why can't we just use feature notes for it, and if we do want a dedicated section why not a feature subsection like features_performance? For example, looking at 2.4 release notes there is not actually a release note on the QPY port to rust besides the prelude (which seems like a big oversight). To me that would have been a features_qpy note that documented the speed up by rewriting it in Rust, including numbers, benchmarking scripts, etc.

I also wonder if we want some rigor in reporting numbers for this section. Besides the obvious question of what is the threshold for what we document in a release note (e.g. is 2% faster in a minority of cases warranted). A lot of release notes are a bit hand wavy if they mention performance. They'll say something like "it's significantly faster now because of a port to Rust". If we want to have a dedicated section on this in release notes I'd think we want some standards about how we report it.

@jakelishman
Copy link
Copy Markdown
Member

This is borne out of an attempt to make "feature" mean "addition", in which case performance doesn't fit. We already do this with the GitHub-space changelog, and even the reno "features" section is called "New features". I feel like your suggestion is to treat "feature" as just meaning "improvement", but imo that's not a particularly useful distinction to readers - you could argue that a bug fix is an "improvement".

To me that would have been a features_qpy note that documented the speed up by rewriting it in Rust, including numbers, benchmarking scripts, etc.

Some of the point of this is that the port to Rust of QPY didn't add any new features (in theory), it just improved existing ones. (Also, somewhat beside the point, but I think benchmarking scripts are probably too much detail for being inline in a release note. Maybe a rough indicative number and a link to a comment in the relevant PR that has scripts/graphs would be a better split.)

I also wonder if we want some rigor in reporting numbers for this section. Besides the obvious question of what is the threshold for what we document in a release note (e.g. is 2% faster in a minority of cases warranted). They'll say something like "it's significantly faster now because of a port to Rust". If we want to have a dedicated section on this in release notes I'd think we want some standards about how we report it.

Personally I'd actually see the point of this section as going the complete other direction: release notes should let users know what changed in case of bugs / notify them of new things to try, so having a dedicated section for performance where we don't have to try to shoe-horn it into the "it's a new performance feature and here's the number to justify it" format will make it easier to have one-line informal notes about it, and I think we should do that fairly liberally on any patch that modified code with the intent of making it faster, not matter how much.

Release notes aren't the place to write full technical details about everything that changed and the situations that they apply to - imo they're the overview to let users know what changed, and include links to more detail if they want to follow up on the technicalities.

@eliarbel
Copy link
Copy Markdown
Member Author

Like Jake I also think of "features" as functional changes and to this end a performance category seems distinct to me. If we continue with this thinking, would it help to stress the difference more explicitly, say in the features part in releasenotes/config.yaml?

@mtreinish
Copy link
Copy Markdown
Member

Release notes aren't the place to write full technical details about everything that changed and the situations that they apply to - imo they're the overview to let users know what changed, and include links to more detail if they want to follow up on the technicalities.

The problem is that the details should exist somewhere, and in my experience they don't exist in most cases unless you require it somewhere. Or if they do how things are measured is inconsistent and not clear. You and I tend to put detailed performance numbers in PR comments or PR summaries (respectively) so it's easy to link to that, but that's far from universal. How many PRs are we going to document in this way where there are not any hard numbers backing it up and just an intent to speed it up? Even if it's me sharing a big table of asv results in a PR comment, that's really opaque to someone that's never seen our asv numbers and there is a lot of shared context assumed there.

I mean we both acknowledge disagreeing on the scope of release notes. In my mind it is the detailed documentation of the release. The release notes should cover everything user facing we included in the release. This is what differentiates the release notes from the changelog which is just a PR summary line and a link to the PR. I think it's where we should put all the details about what changed, why it changed, and how you use the changes. This isn't in lieu of having real documentation but we should have sufficient details in there so someone understands the changes made in the release. In my experience if you have to click through to multiple links that just makes it hard for people to find and doesn't have any consistency. Especially in this case if there is an evolution of performance in the development of the PR that's tracked through successive comments that github hides by default. But I'm fine not seeing eye to eye on that and trying it a different way. But if we are going to start calling out performance I want people to be able to understand why we're calling it out and what they can expect and I worry just saying something like "random circuit synthesis is faster now in some scenarios see: #123456667 for details" does not help with that understanding.

@mtreinish
Copy link
Copy Markdown
Member

Like Jake I also think of "features" as functional changes and to this end a performance category seems distinct to me. If we continue with this thinking, would it help to stress the difference more explicitly, say in the features part in releasenotes/config.yaml?

If that is the intent of the move I think tweaking the template language to make it clear that "features" === "additions" or however you want to word it would help. Right now it doesn't define what a feature means and it's left to interpretation.

@eliarbel
Copy link
Copy Markdown
Member Author

If that is the intent of the move I think tweaking the template language to make it clear that "features" === "additions" or however you want to word it would help. Right now it doesn't define what a feature means and it's left to interpretation.

How is this language as an addition on top of the features section in the release notes template file:

A feature is a new user-visible functionality. It can come in the form of a new function, class, parameter, environment variable or any other user accessible and publicly documented endpoint. In addition, new capabilities (for example "Qiskit can now compile to Clifford+T"), newly supported workflows and new types of language interoperability are considered features.

@eliarbel
Copy link
Copy Markdown
Member Author

Jake, Matthew: I've added the additional explanation to the features section in the template file. Is there anything else you'd like to discuss or have addressed here?

Copy link
Copy Markdown
Member

@jakelishman jakelishman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this, thanks Eli. I'll let Matt enqueue in case he wants to continue the discussion.

@mtreinish mtreinish added this pull request to the merge queue Apr 30, 2026
Merged via the queue into Qiskit:main with commit d3199b0 Apr 30, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Changelog: None Do not include in the GitHub Release changelog.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants